Machine Learning and Natural Language Processing in Stock Prediction

In this thesis, we first study the two ill-posed natural language processing tasks related to stock prediction, i.e. stock movement prediction and financial document-level event extraction. While implementing stock prediction and event extraction, we encountered difficulties that could be resolved by utilizing out-of-distribution detection. Consequently, we presented a new approach for out-of-distribution detection, which is the third focus of this thesis. First, we systematically build a platform to study the NLP-aided stock auto-trading algorithms. Our platform is characterized by three features: (1) We provide financial news for each specific stock. (2) We provide various stock factors for each stock. (3) We evaluate performance from more financial-relevant metrics. Such a design allows us to develop and evaluate NLP-aided stock auto-trading algorithms in a more realistic setting. We also propose a system to automatically learn a good feature representation from various input information. The key to our algorithm is a method called semantic role labelling Pooling (SRLP), which leverages Semantic Role Labeling (SRL) to create a compact representation of each news paragraph. Based on SRLP, we further incorporate other stock factors to make the stock movement prediction. In addition, we propose a self-supervised learning strategy based on SRLP to enhance the out-of-distribution generalization performance of our system. Through our experimental study, we show that the proposed method achieves better performance and outperforms all strong baselines’ annualized rate of return as well as the maximum drawdown in back-testing. Second, we propose a generative solution for document-level event extraction that takes into account recent developments in generative event extraction, which have been successful at the sentence level but have not yet been explored for document-level extraction. Our proposed solution includes an encoding scheme to capture entity-to-document level information and a decoding scheme that takes into account all relevant contexts. Extensive experimental results demonstrate that our generative-based solution can perform as well as state-of-theart methods that use specialized structures for document event extraction. This allows our method to serve as an easy-to-use and strong baseline for future research in this area. Finally, we propose a new unsupervised OOD detection model that separates, extracts, and learns the semantic role labelling guided fine-grained local feature representation from different sentence arguments and the full sentence using a margin-based contrastive loss. Then we demonstrate the benefit of applying a self-supervised approach to enhance such global-local feature learning by predicting the SRL extracted role. We conduct our experiments and achieve state-of-the-art performance on out-of-distribution benchmarks.Thesis (Ph.D.) -- University of Adelaide, School of Computer and Mathematical Sciences, 202

Semantic Role Labeling Guided Out-of-distribution Detection

Identifying unexpected domain-shifted instances in natural language processing is crucial in real-world applications. Previous works identify the OOD instance by leveraging a single global feature embedding to represent the sentence, which cannot characterize subtle OOD patterns well. Another major challenge current OOD methods face is learning effective low-dimensional sentence representations to identify the hard OOD instances that are semantically similar to the ID data. In this paper, we propose a new unsupervised OOD detection method, namely Semantic Role Labeling Guided Out-of-distribution Detection (SRLOOD), that separates, extracts, and learns the semantic role labeling (SRL) guided fine-grained local feature representations from different arguments of a sentence and the global feature representations of the full sentence using a margin-based contrastive loss. A novel self-supervised approach is also introduced to enhance such global-local feature learning by predicting the SRL extracted role. The resulting model achieves SOTA performance on four OOD benchmarks, indicating the effectiveness of our approach. Codes will be available upon acceptance

Stock Market Prediction via Deep Learning Techniques: A Survey

The stock market prediction has been a traditional yet complex problem researched within diverse research areas and application domains due to its non-linear, highly volatile and complex nature. Existing surveys on stock market prediction often focus on traditional machine learning methods instead of deep learning methods. Deep learning has dominated many domains, gained much success and popularity in recent years in stock market prediction. This motivates us to provide a structured and comprehensive overview of the research on stock market prediction focusing on deep learning techniques. We present four elaborated subtasks of stock market prediction and propose a novel taxonomy to summarize the state-of-the-art models based on deep neural networks from 2011 to 2022. In addition, we also provide detailed statistics on the datasets and evaluation metrics commonly used in the stock market. Finally, we highlight some open issues and point out several future directions by sharing some new perspectives on stock market prediction

Surface Defects Recognition of Wheel Hub Based on Improved Faster R-CNN

Machine vision is one of the key technologies used to perform intelligent manufacturing. In order to improve the recognition rate of multi-class defects in wheel hubs, an improved Faster R-CNN method was proposed. A data set for wheel hub defects was built. This data set consisted of four types of defects in 2,412 1080 × 1440 pixels images. Faster R-CNN was modified, trained, verified and tested based on this database. The recognition rate for this proposed method was excellent. The proposed method was compared with the popular R-CNN and YOLOv3 methods showing simpler, faster, and more accurate defect detection, which demonstrates the superiority of the improved Faster R-CNN for wheel hub defects

A Fault-Signal-Based Generalizing Remaining Useful Life Prognostics Method for Wheel Hub Bearings

The goal of this work is to improve the generalization of remaining useful life (RUL) prognostics for wheel hub bearings. The traditional life prognostics methods assume that the data used in RUL prognostics is composed of one specific fatigue damage type, the data used in RUL prognostics is accurate, and the RUL prognostics are conducted in the short term. Due to which, a generalizing RUL prognostics method is designed based on fault signal data. Firstly, the fault signal model is designed with the signal in a complex and mutative environment. Then, the generalizing RUL prognostics method is designed based on the fault signal model. Lastly, the simplified solution of the generalizing RUL prognostics method is deduced. The experimental results show that the proposed method gained good accuracies for RUL prognostics for all the amplitude, energy, and kurtosis features with fatigue damage types. The proposed method can process inaccurate fault signals with different kinds of noise in the actual working environment, and it can be conducted in the long term. Therefore, the RUL prognostics method has a good generalization

Astock: A New Dataset and Automated Stock Trading based on Stock-specific News Analyzing Model

Natural Language Processing(NLP) demonstrates a great potential to support financial decision-making by analyzing the text from social media or news outlets. In this work, we build a platform to study the NLP-aided stock auto-trading algorithms systematically. In contrast to the previous work, our platform is characterized by three features: (1) We provide financial news for each specific stock. (2) We provide various stock factors for each stock. (3) We evaluate performance from more financial-relevant metrics. Such a design allows us to develop and evaluate NLP-aided stock auto-trading algorithms in a more realistic setting. In addition to designing an evaluation platform and dataset collection, we also made a technical contribution by proposing a system to automatically learn a good feature representation from various input information. The key to our algorithm is a method called semantic role labeling Pooling (SRLP), which leverages Semantic Role Labeling (SRL) to create a compact representation of each news paragraph. Based on SRLP, we further incorporate other stock factors to make the final prediction. In addition, we propose a self-supervised learning strategy based on SRLP to enhance the out-of-distribution generalization performance of our system. Through our experimental study, we show that the proposed method achieves better performance and outperforms all the baselines' annualized rate of return as well as the maximum drawdown of the CSI300 index and XIN9 index on real trading. Our Astock dataset and code are available at https://github.com/JinanZou/Astock

Magnetic nanoparticles coated with maltose-functionalized polyethyleneimine for highly efficient enrichment of N-glycopeptides

Hydrophilic interaction chromatography (HILIC) adsorbents have drawn increasing attention in recent years due to their high efficiency in N-glycopeptides enrichment. The hydrophilicity and binding capacity of HILIC adsorbents are crucial to the enrichment efficiency and mass spectrometry (MS) detection sensitivity of N-glycopeptides. Herein, magnetic nanoparticles coated with maltose-functionalized polyethyleneimine (Fe3O4-PEI-Maltose MNPs) were prepared by one-pot solvothermal reaction coupled with "click chemistry" and utilized for N-glycopeptides enrichment. Owing to the presence of hydrophilic and branched polyethyleneimine, the amount of immobilized disaccharide units was improved about four times. The N-glycopeptides capturing capacity was about 150 mg/g (IgG/MNPs) and the MS detection limitation as low as 115 fmol for IgG and 85% average enrichment recovery were feasibly achieved by using this hybrid magnetic adsorbent. Finally, 1237 unique N-glycosylation sites and 1567 unique N-glycopeptides from 684 N-glycoproteins were reliably characterized from 60 mu g protein sample extracted from mouse liver. Therefore, this maltose-functionalized polyethyleneimine coated adsorbent can play a promising role in highly efficient N-glycopeptides enrichment for glycoproteomic analyses of complex protein samples. (C) 2015 Elsevier B.V. All rights reserved

Establishing a Urine-Based Biomarker Assay for Prostate Cancer Risk Stratification

One of the major features of prostate cancer (PCa) is its heterogeneity, which often leads to uncertainty in cancer diagnostics and unnecessary biopsies as well as overtreatment of the disease. Novel non-invasive tests using multiple biomarkers that can identify clinically high-risk cancer patients for immediate treatment and monitor patients with low-risk cancer for active surveillance are urgently needed to improve treatment decision and cancer management. In this study, we identified 14 promising biomarkers associated with PCa and tested the performance of these biomarkers on tissue specimens and pre-biopsy urinary sediments. These biomarkers showed differential gene expression in higher- and lower-risk PCa. The 14-Gene Panel urine test (PMP22, GOLM1, LMTK2, EZH2, GSTP1, PCA3, VEGFA, CST3, PTEN, PIP5K1A, CDK1, TMPRSS2, ANXA3, and CCND1) was assessed in two independent prospective and retrospective urine study cohorts and showed high diagnostic accuracy to identify higher-risk PCa patients with the need for treatment and lower-risk patients for surveillance. The AUC was 0.897 (95% CI 0.939–0.855) in the prospective cohort (n = 202), and AUC was 0.899 (95% CI 0.964–0.834) in the retrospective cohort (n = 97). In contrast, serum PSA and Gleason score had much lower accuracy in the same 202 patient cohorts [AUC was 0.821 (95% CI 0.879–0.763) for PSA and 0.860 (95% CI 0.910–0.810) for Gleason score]. In addition, the 14-Gene Panel was more accurate at risk stratification in a subgroup of patients with Gleason scores 6 and 7 in the prospective cohort (n = 132) with AUC of 0.923 (95% CI 0.968–0.878) than PSA [AUC of 0.773 (95% CI 0.852–0.794)] and Gleason score [AUC of 0.776 (95% CI 0.854–0.698)]. Furthermore, the 14-Gene Panel was found to be able to accurately distinguish PCa from benign prostate with AUC of 0.854 (95% CI 0.892–0.816) in a prospective urine study cohort (n = 393), while PSA had lower accuracy with AUC of 0.652 (95% CI 0.706–0.598). Taken together, the 14-Gene Panel urine test represents a promising non-invasive tool for detection of higher-risk PCa to aid treatment decision and lower-risk PCa for active surveillance